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Abstract — This paper presents an efficient algorithm for 
robust network reconstruction of Linear Time-Invariant (LTI) 
systems in the presence of noise, estimation errors and unmod- 
elled nonlinearities. The method here builds on previous work 
[1] on robust reconstruction to provide a practical implementa- 
tion with polynomial computational complexity. Following the 
same experimental protocol, the algorithm obtains a set of 
structurally-related candidate solutions spanning every level 
of sparsity. We prove the existence of a magnitude bound 
on the noise, which if satisfied, guarantees that one of these 
structures is the correct solution. A problem-specific model- 
selection procedure then selects a single solution from this set 
and provides a measure of confidence in that solution. Extensive 
simulations quantify the expected performance for different 
levels of noise and show that significantly more noise can be 
tolerated in comparison to the original method. 

I. Introduction 

An active topic of dynamical systems research and an 
ubiquitous problem is that of network reconstruction, or 
inferring information about the structural and dynamical 
properties of a networked system [l]-[7]. This problem is 
motivated by a diverse range of fields where an unknown 
system can be described by interconnecting subsystems 
acting between accessible system states. By perturbing the 
system and measuring these states, we seek to understand the 
nature and dynamics of their interactions. In general, this is 
an underdetermined problem and is further complicated by 
the fact that not all states may be available for measurement 
and that there may also be additional, unknown states. Here 
we focus on noisy LTI systems and seek to obtain reliable 
structural and dynamic information involving the measured 
states, whilst leaving the hidden states unrestricted. 

One prominent example from Systems Biology is the 
identification of gene regulatory networks, which describe 
the interactions between genes via biochemical mechanisms. 
Despite the stochastic and nonlinear nature of biological 
systems, a linear-systems description provides an appealing 
and tractable option. Measurements of the expression levels 
of individual genes are routinely available (for example from 
microarray experiments) and a number of these are taken 
as system states. Additional 'hidden' states are required to 
accurately model the effect of protein and other metabolic 
interactions, any process with higher than first-order dy- 
namics and any important genes that are not selected as 
system states. In such applications, the assumption of full- 
state measurement may well lead to incorrect conclusions. 

Nevertheless, there are a number of published methods to 
obtain a representative linear model from full state measure- 
Department of Engineering, University of Cambridge, UK. jdph34, 
yy311, jmg77)@cam. ac.uk 



ments, for example [2] using Linear Matrix Inequalities, [3] 
using a 1-norm residual and [4] using compressive sensing. 
There are many other approaches to network reconstruction, 
for example Bayesian [5], information theoretic [6] and 
Boolean [7], in all of which the solution is biased, most 
commonly towards sparsity, to compensate for the lack of 
information. The vice of all these methods is that if the 
true solution is not sparse, a sparse solution will be obtained 
anyway. 

Dynamical structure functions were introduced in [8] as 
a means of representing the structure and dynamics of an 
LTI system at a resolution consistent with the number of 
measured states. Exactly how much additional information 
is required to reconstruct a network from the system transfer 
matrix could then be quantified. In particular, without extra 
information, it is possible to obtain a solution that justifies 
any prior assumption. Applications of dynamical structure 
functions include multi-agent systems [9], [10] in addition to 
network reconstruction [11], where necessary and sufficient 
conditions were given for exact reconstruction from the 
transfer matrix. 

In [1], this approach was made robust to uncertainty in the 
transfer matrix by obtaining dynamical structure functions 
that are closest, in some sense, to the data. Since the dynami- 
cal structure function is no longer unique, it is also necessary 
to estimate Boolean network structure, that is, an unweighted 
directed graph of causal connections. The approach taken 
was to calculate the optimum dynamical structure function 
for every possible Boolean network for a given number of 
measured states, p, then use a model selection technique to 
select the best estimate. This approach suffers chiefly from 
a high computational complexity of (9(2''), which limits its 
use to relatively small networks. 

Here we present an extension of [1] that does not require 
all of the Boolean network structures to be considered. In 
fact, we obtain a set of p 2 — p + 1 candidate structures by ju- 
diciously removing links from the fully-connected structure, 
with complexity in the number of structures that must be 
considered of 0(p 3 ). This set contains one structure for every 
level of sparsity and hence there is no prior bias towards 
sparser solutions. 

Section [TT] defines dynamical structure functions and states 
necessary previous results. The main result of a polynomial- 
time reconstruction algorithm is then presented in Section 
Hill Section [IV] introduces a new model selection procedure. 
Section [V] then compares the performance of two variants 
of the method introduced here with that of [1] in extensive 
random simulations. Conclusions and an outline of future 
work are given in Section [VTl 



II. Dynamical Structure 

We consider Linear Time-Invariant (LTI) systems of the 
following form: 

x = Ax + Bu 

(1) 

y = [ I ]x 

where x e R" is the full state vector, u e R m is the vector 
of inputs, y e R p is the vector of measured states, with < 
p < n, and / is the p x p identity matrix. That is, we assume 
that some of the states are directly measured and some are 
not. It is also possible to consider a more general form of 
C matrix, as in [12]. By eliminating the hidden states, the 
dynamical structure function representation can be derived 
(see [11]) as: 

Y = QY + PU (2) 

where Y and U are the Laplace transforms of y and u and 
Q and P are strictly proper transfer matrices. The hollow 
matrix Q is the Internal Structure and dictates direct causal 
relationships between measured states, which may occur 
via hidden states. The matrix P is the Control Structure 
and similarly defines the relationships between inputs and 
measured states that are direct in the sense that they do not 
occur via any other measured states. The dynamical structure 
function is defined as (Q,P) and the Boolean dynamical 
structure is defined as the Boolean matrices S(Q), S(P) that 
have the same zero elements as Q and P. 

A. Dynamical Structure Reconstruction 

The problem of network reconstruction was cast in [11] 
as a two-stage process, whereby the transfer matrix G is first 
obtained from input-output data by standard system identi- 
fication techniques and the dynamical structure function is 
then obtained from G. Here we state the main results for 
the second stage of this process. The dynamical structure 
function for a given state-space realisation is unique, and 
related to the transfer matrix, G, as follows: 

G = (I-Q)- 1 P (3) 

Whilst every (Q,P) uniquely specifies a G, there are many 
such possible (Q,P)s for any given G and a dynamical 
structure function is said to be consistent with a given 
transfer matrix if, and only if, there exists a state-space 
realisation for which the dynamical structure function (at the 
considered resolution) satisfies OJ. Exact reconstruction is 
therefore possible if, and only if, there is only one (Q, P) that 
is consistent with G, which requires some a priori knowledge 
of (Q, P). Corollary [T]defines an experimental protocol under 
which this condition is met. 

Corollary 1 ( [11]). If G is full rank and p inputs are 
applied, where there are p measured states and each input 
directly and uniquely affects only one measured state (such 
that the matrix P can be made diagonal), then (Q,P) can be 
recovered exactly from G. 

Essentially the zero elements of P comprise sufficient knowl- 
edge to obtain (Q, P) directly from (01. Here we assume that 



the conditions of the above corollary have been met, which 
ensures solution uniqueness when G is known perfectly. 

B. Robust Dynamical Structure Reconstruction 

In practice, the transfer matrix G will be a noisy estimate 
of the true transfer matrix, as considered in [1]. As a result, 
the dynamical structure function obtained directly from G 
may not be the best approximation of that of the true 
system. The noise is expressed as a perturbation on the 
true transfer matrix G,, for example as feedback uncertainty: 
G, — (I + A)~ l G for some transfer matrix A. Since G will 
typically admit a fully-connected dynamical structure func- 
tion, a strategy to estimate the Boolean dynamical structure 
consistent with G, is required. 

Consider the i th Boolean dynamical structure function and 
denote (Qi,Pi) as a dynamical structure function with this 
Boolean structure. We can relate G to a transfer matrix G, 
that is consistent with (Qt,Pd by G = (/ + A,)G,- for some 
transfer matrix A,-. In this case, <£3j becomes: (/ + A,)~'G = 
(7 - QiT l Pi, which can be rearranged as: 

A i = GPr\l-Q i )-I 
= GXi-I 

where X, = Pt 1 (I - g,) and has the same non-diagonal 
Boolean structure as Q;. By minimising some norm of A,, 
with respect to X,, we can obtain the Xj and hence the 
corresponding dynamical structure function that is consistent 
with the closest transfer matrix to G. 

The approach of [1] was to minimise ||A,|| for every 
possible Boolean Q; (of which there are TP ~ p since Q has 
p 2 - p degrees of freedom), then use Akaike's Information 
Criterion (AIC) [13] to select a solution by penalising the 
number of nonzero elements in Q. Specifically, let Xi be the 
set of all X that satisfy the constraints of the i th Boolean 
X, and minimise the Frobenious norm over s = ja> of A,- as 
follows: 

6] = inf \\GX - I\% (5) 

XeX, 

to obtain a measure of the smallest distance 6j from G to G,-. 
This choice of norm allows the problem to be cast as a least 
squares optimisation, and we denote this method Mq. 



Algorithm Mq 

for i = 1 -> 2'' 2 -p do 
$ 2 i = M XeXi \\GX - I\\ 2 P 
X J =aiginf Z6 x f ||GX-/||! 

end for 

Apply AIC to the set {X,}. 



Hence every Boolean structure S(Xj) can be associated 
with a distance measure 6j from (|5]l and a dynamic structure 
Xi, which is the corresponding minimising argument. The 
principal problem with this approach is that the computa- 
tional complexity is dominated by the number of optimisa- 
tions that must be performed, which can be reduced to p2 p by 
performing the optimisation of the columns of X separately. 



III. Main Result 

A. A Polynomial Time Algorithm 

Here we propose an algorithm with polynomial complexity 
to estimate the dynamical structure function of G,, under the 
conditions of Corollary Q] First, an iterative procedure is used 
to obtain a set §, containing Boolean internal structures, with 
one structure (S J ) for each level of sparsity (j links). Then 
a model selection procedure is applied to this reduced set 
to select a single solution. This method is denoted M2 and 
defined as follows: 

Algorithm M2 

2 

Set S p p as the fully-connected structure, 
for j - p 2 - p — > 1 do 

Remove one link of S J at a time to obtain a set of j 
structures with j — 1 links and calculate for each of 
these structures. 

Set 5 1 - 7-1 as the minimum-^-' -1 structure, 
end for 

Set S° as the decoupled structure. 

Apply a model selection procedure to the set S = {S^}. 



Fig. Q] illustrates this procedure for a p = 3 example. The 
number of structures that must be considered is exactly 
j(p 4 - 2p 3 + 2p 2 - p) + 1. By optimising the columns 
of X separately, the overall number of optimisations that 
must be performed is of the order 0(p 3 ), and since the 
complexity of the optimisations is also polynomial, the 
overall computational complexity is polynomial. 

The reasoning behind this approach is that the fully- 
connected structure is composed of all the links belonging 
to the true structure plus extra links that afford it a smaller 6 
by better modelling the noise. It is intuitive that if the level 
of noise is not too high, removing false links should have a 
smaller effect on 5 than removing true links, in which case all 
the false links would be removed first and the true structure 
would be encountered. In fact, we will show that if the noise 
is sufficiently small (in some norm) then this is always the 
case. 

B. An Intermediate Algorithm 

We note that an intermediate algorithm can be defined, 
with the same two-stage approach of M2 but making use 
of all 7y~ p Boolean structures and hence with exponential 
complexity. Rather than obtain the set S iteratively, this 
method obtains a set T = {T^}, where each T-i is the 
minimum-(5 Boolean structure over all Boolean structures 
with j links. 

Note that the set T contains the minimum-AIC structures 
for each level of sparsity, and hence Method Mo is only able 
to select a solution from this set. If the true Boolean structure 
is not in the set T, then neither Mq nor M\ can obtain the 
correct structure, from which we can define: 

Definition 1 (M\ Solvability). A given reconstruction prob- 
lem is solvable by Method Mi (and Mo) if the true Boolean 
internal structure is in the set T. 




Fig. 1: An example set S for Method M 2 for a network with three 
measured states. Solid arrows denote true links and dashed arrows 
denote false links. Starting with the fully-connected structure (S 6 ), 
one network is obtained for each level of sparsity by successively 
removing the link which results in the smallest increase in d. A 
model selection technique is then used to select a single solution 
from this set. In this example, the true network is S 3 . 



Algorithm Mi 

for j = -> p 2 - p do 

Compute 5' for all Boolean internal structures with j 
links. 

Set as the minimum-^- 7 structure, 
end for 

Apply a model selection procedure to the set T = {J- 7 }. 



Therefore, if a problem is Mi solvable, Method M\ will 
always obtain a set T that contains the true Boolean structure. 
This step has separated the uncertainty inherent in the 
problem due to noise from that due to the model selection 
process. If the model selection stage is not correct but the 
problem is Mi solvable, then we have a relatively small set 
of candidate structures, one of which is the true structure. 

In summary, Mo considers all 2 P ~ p Boolean structures 
and selects a single solution; Mi considers all 2 P ~~ P Boolean 
structures, selects the best-fitting structure for each number 
of links to form a subset of p 2 - p + 1 structures and then 
selects a single solution from this subset; M2 iteratively finds 
a set of p 2 - p + 1 Boolean structures and selects a single 
solution from this set, without having to consider all possible 
structures. We will next consider a sufficient condition under 
which a problem can be solved by M%, and it will be seen 
that this condition is also sufficient for the problem to be M\ 
solvable. 

C. Solvability Conditions for M2 

We define the solvability of M2 as follows: 

Definition 2 (M2 Solvability). A given reconstruction prob- 
lem is solvable by Method M2 if the true Boolean internal 
structure is in the set S. 

First it is noted that the false links of the fully-connected 
structure can be removed in any order to obtain the true 
structure, so the path to the true structure is not unique. A 
set of allowable Boolean structures can be defined as all 
those that contain at least all of the true links. All other 
structures with no fewer links than the true structure are non- 



allowable, as to encounter any one of these will mean that 
the true structure will not be reached. Fig. [2] shows 6 values 
plotted against number of links for an example allowable set 
plus valid 'paths' which may be taken between structures 
by removing one link. The following Lemma provides a 
sufficient condition for Mo solvability: 

Lemma 1. A given reconstruction problem is solvable by 
Method M2 if all allowable structures have smaller 6 than 
all non-allowable structures with the same number of links. 

Proof. For every allowable structure that is not the true 
structure, another allowable structure can always be obtained 
by removing one link, so a path always exists to another 
allowable structure. If the condition of the Lemma holds, at 
every stage in the M2 algorithm an allowable structure will 
be selected in preference to a non-allowable structure until 
the true structure is reached. ■ 

This is equivalent to all non-allowable structures being 
within the open shaded region in Fig. [2] Note that if the 
condition of Lemma Q] holds, this also implies that the 
problem is Mi solvable. We now derive, for each Boolean 
structure, an expression for the deviation of the 5 value in 
the case of noise from its nominal (no noise) value. The level 
of noise such that the condition of Lemma Q] is met can then 
be characterised. 

From (0, the value of 5, for the i Boolean structure is 
given by: 

6 2 =M\\GX-lf P 

2 (6) 
= inf \\AY - b\\ 2 7 

FeJ/, 

where A - I ® G, Y = vec(X) and b = vec(7) where I is 
the p x p identity matrix, ® is the Kronecker product and 
vec(-) is the vectorization operator. The set J/, contains a 
vector Y = \ec{X) for each element X e Consider a 
subset of Boolean internal structures, which can be obtained 
from the fully-connected structure by constraining at most 
one element in each column of X to be zero. The following 
Lemma relates the 6 values of these structures to that of the 
fully-connected structure. 

Lemma 2. For any Boolean structure S(Xf) with no more 
than one zero element in each column, the value of Si is 
related to that of the fully-connected structure as follows: 

for all elements j of Y that are constrained in the i th 
structure. Here Yf u u — vec(X/„//) for the fully-connected 
structure Xf u ii, which has 6 value Sf„u, A + := (A'A)~' and ' 
denotes complex conjugate transpose. 

The proof is given in Appendix|A]and follows from writing 
|6) as a constrained optimisation problem. In particular, this 
special case applies to all structures with only one element in 
total constrained to be zero. We now consider directly how 
a perturbation on the true transfer matrix G, affects Si and 
apply a recursive argument to make use of this special case. 
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Fig. 2: An example problem showing S values plotted against 
number of links for the allowable set of Method Mi. Allowable 
Boolean structures (those which contain at least all of the true 
links) are marked by an x. Allowable paths (those that may be 
taken between allowable structures by removing a single link) are 
shown as solid lines. The problem is M2 solvable, by LemmafJJ if 
all non-allowable structures are within the open shaded region. 



Write the feedback uncertainty in G, as A = eAo, where 
e = maXy{|[A.(i,/)IL}, such that G = (7 + eA )G,. Now 
given a perturbation eAo, parameterise the solution for the S 
value of the i' h structure by e as 5,(e). We first consider the 
structures obtained by constraining one link of Yf u u. 

Lemma 3. Given an estimate of the true transfer matrix G t 
as G — (I + eAo)G t , the smallest distance Si from Gi to G for 
every Boolean structure S(Xi) with only one zero element is 
given by: 

d 2 (e) = d 2 (0) + S 2 full (e)+f i (e) (8) 

where f(e) is continuous and satisfies f(0) = 0. 

The proof is given in Appendix 151 and essentially consists 
of expressing the uncertainty in S 2 (e) directly using Lemma 
[2] For any perturbation eAo on G t , the corresponding devi- 
ation of df(e) from its nominal (e = 0) value is therefore 
given by 5 2 ull (e) + f(e). The following Lemma extends this 
to describe the deviation of S 2 (e) for a general X,. 

Lemma 4. Given G — (I + eAo)G,, the smallest distance Si 
from Gj to G for every Boolean structure S(Xi) is given by: 



S 2 (e) = S 2 (0) + S 2 j (e) + f ij (e) 



(9) 



for any Boolean structure S(Xj) from which B(Xi) can be 
obtained by constraining one link. The function fj(e) is 
continuous and satisfies fj(0) = 0. 

The proof is given in Appendix [Cj where Xj is treated 
as the solution to a new, unconstrained problem and hence 
Lemma [3] can be applied. Using the results of Lemmas [3] 
and [4] it is possible to compute the deviation of every 5,(e) 
from its nominal (5,(0) value. By considering the difference 
between the deviations of any pair of allowable and non- 
allowable structures as a function of e, it can be seen that 
for sufficiently small e, the condition of Lemma Q] is always 
met and the problem is therefore M2 solvable. 



Theorem 1. Given G — (I + eAo)G t , where G t is the true 
transfer matrix, there exists an r > such that for all e < r, 
the problem is always solvable by Method Mi- Specifically, 
the difference between 6„ for any non-allowable structure 
and 6 a for any allowable structure with the same number of 
links is given by: 

6 2 „(e) - S 2 a (e) = B„ + F na (e) (10) 

where the constant B n satisfies B n > and \F na (e)\ < B„ for 
all e < r. 

The proof is given in Appendix [D] and comprises a 
recursive application of Lemma 0] to assess the solvability 
condition of Lemma Q] This result proves that every problem 
is M2 solvable for sufficiently small e, and given G, and Ao 
it is possible to calculate a bound r on the size of this e. 
Method M2 is then guaranteed to encounter the true structure 
by successively removing links from the fully-connected 
structure. 

Remark 1 (Repeated Experiments). We note that multiple 
estimates of G t are readily incorporated by simply replacing 
G with a block vector of these estimates. 

Remark 2 (Prior Information). If something is known about 
the true Boolean structure, this information can easily be 
taken into account in Algorithm M2 by restricting the number 
of structures that must be considered. This will then reduce 
the computational complexity. 

Remark 3 (Steady-State Reconstruction). As in [1 ], if only 
steady-state measurements are available, we can reconstruct 
the steady-state dynamical structure function from G(0). 
Complications arise in the case of transfer functions with 
zero steady-state gain, but otherwise the results of Section 
\nl\can be directly applied to steady-state reconstruction. In 
particular, the bound of Theorem\l\is tighter and significantly 
easier to compute. 

IV. A New Model Selection Approach 

Here we introduce a method of selecting a candidate 
Boolean structure from the set S of Method M%, as an 
alternative to AIC. Our method does not directly penalise 
model complexity but rather seeks to locate the correct level 
by identifying the subsequent loss of information as the 
complexity is further reduced. Denote the 5 value of S ' e § 
as 6' and the number of links of the true structure as i,. Note 
that for no noise, 6' — for i, < i < p 2 - p and 5' > for 
i < i t , since the solution is unique. Now define the following 
normalised derivative of 6: 

(?'-' -6? 2 
d t = ■_! , i = \,...,p -p (11) 

where d' Q := and < d\ < 1 since < 6' < &~ x . For 
low levels of noise, 5' is close to zero for all i > i t and 
increases significantly for i < i,, and it is this increase in 6 l 
that we seek to detect. In fact, as the noise level approaches 
zero, d\ — > 1 for i - i,. Taking a form of second derivative, 



defined as follows, was found heuristically to improve the 
distinction of the true structure: 

d" = max{c/; - d' i+l , 0), i = 0, . . . ,p 2 - p (12) 

where d", , := and < d" < 1 . The candidate solution is 

p—p+i 1 

then selected as S 1 where j = argmax,{c/ ( "} and a measure of 
the confidence in this selection is given by max/faf") e [0, 1]. 

Remark 4 (A Single Solution). Whilst it is useful to obtain 
a single solution for comparative simulations, in practice, a 
more prudent approach is advised. For example, the relative 
merit of each of the structures in the solution sets of M\ or 
M2 could be considered. 

V. Simulations 

Methods M\ and M2 introduced here were compared in 
simulation with Mq from [1] on steady-state network recon- 
struction of a large number of linear test networks. Networks 
with three measured states and up to three hidden states were 
considered; for each of the 64 possible Boolean network 
structures with three measured states, 300 random, stable 
linear systems were generated. For each test system and for 
a range of noise variance from 10~ 5 - 10, three experimental 
estimates of G(0) were obtained as G(0) = (/ + A)G ; (0), 
where G r is the true transfer matrix and the elements of A 
were sampled from a zero-mean normal distribution. These 
estimates of G(0) were then used by each method to attempt 
to obtain the correct steady-state network structure, given no 
other information about the true network. In total, 963, 900 
network reconstruction problems were considered. 

Fig. [3] shows the average number of networks that were 
correctly identified by each method, for each level of noise, 
as a percentage of the total number of reconstruction prob- 
lems attempted. Some variation in performance was observed 
between different network structures (some were harder to 
identify than others) but the results of Fig. [3] are representa- 
tive of the performance of each of the methods. Also shown 
in Fig. [3] is the Mi solvable limit, which is the percentage 
of reconstruction problems that were M\ solvable. 

The most important result is that the performances of M\ 
and M2 are almost identical, validating the use of M2 for 
this problem class. Only 0.37% of problems could be solved 
by Mi but not by M2, which is certainly justified by the 
significant reduction in computational complexity. The level 
of noise required for M2 to fail is apparently similar to that 
required for M\ to fail. In addition, M\ and M2 (both using 
the model selection procedure of Section |IVt consistently 
outperform Mq for all noise levels. For example, for a noise 
variance of 10~ 3 , approximately 60% of problems could be 
solved by Mo, whereas 90% could be solved by Mi and M2. 

Figure [4] shows the percentage of reconstruction problems 
for which the set of possible Boolean solutions obtained by 
each method contained the true Boolean structure. Again the 
results for methods Mi and M2 are almost identical. For a 
noise variance of 10~ 2 , only approximately 30% of problems 
could be solved by Mq, whereas for Mi and M2 we could 
be 90% certain that the correct Boolean structure is in the 
set. 
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Fig. 3: Percentage of reconstruction problems successfully solved 
by each method. M is the method from [1] with exponential 
complexity; Mi is an intermediate method also with exponential 
complexity; M 2 is the polynomial complexity method. Also shown 
is the Mi solvable limit, which is the percentage of reconstruction 
problems that were solvable by M\ (and Mo). The difference 
between this curve and that of Mo and Mi is due to the incorrect 
solution being chosen in the model selection stage. 



VI. Conclusions 

A. Summary 

This paper introduces an algorithm with polynomial com- 
plexity that robustly reconstructs the structure and dynam- 
ics of an unknown LTI network in the presence of noise 
and unmodelled nonlinearities. Specifically, we estimate the 
dynamical structure function from a noisy estimate of the 
system transfer matrix. Rather than seek to obtain a sparse 
solution, we consider a set of solutions spanning all levels of 
sparsity and then select a solution from this set. Following a 
certain experimental protocol, we prove that every such prob- 
lem is solvable by our method if the magnitude of the noise 
is sufficiently small, where the size of this bound depends 
on the properties of the system in question. The expected 
performance of this method is assessed in simulation, which 
demonstrates almost no difference in performance between 
the exponential and polynomial complexity versions and also 
shows significant improvements over the previous method. 

B. Future Work 

Future work will consider exactly what properties of the 
true system contribute to the size of the noise bound for that 
system. This may provide insight to develop a necessary and 
sufficient condition for solvability, given a certain type of 
noise perturbation. We also seek to relax the conditions of 
the experimental protocol, which is another limiting factor 
in the size of problem that this approach can handle. 
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Fig. 4: Percentage of reconstruction problems where the solution 
set obtained by each method contained the true solution. M is the 
method from [1] with exponential complexity; Mi is an intermediate 
method also with exponential complexity; M 2 is the polynomial 
complexity method. Note that the solution set of method Mo only 
contains a single solution. 
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Appendix 

A. Proof of Lemma [2] 

Proof. Equation (O can be written as a constrained optimi- 
sation problem as follows: 

5*=M\\AY-b\\l subject to C-Y = (13) 

where C,- contains the relevant columns of the p 2 x p 2 identity 
matrix. The constraints are appended to the cost function by 
a vector of Lagrange multipliers p,\ 



1 f°° 

S 2 = mf—\ \AY - b\ 2 + p'jC'jY dco 



(14) 



C'Y = 



with s = joj for all transfer functions. Solving for Y, and fij 
yields: 

Yi = (A'A)- 1 U'b - \cw\ 

\ 2 I (15) 

Pi = 2(c;(A'A)- 1 C / )" 1 qiA'A^A'b 

The following abbreviations are made to simplify notation: 
A + = (A'A)-\ Di = C,'A + C, and E t = QTJ^C-A" 1 ". It is noted 
that the fully-connected (unconstrained) solution is Yf u u = 
A + A'b and the corresponding df u ii given by: 



1 r°° 



(7 - AA + A') b dco 



(16) 



Using (fT4T > and ( fTBI l. for the /''' Boolean structure the solution 
is Yi - A + (I - Ei) A'b and the optimum 5, is given as 
follows: 



dco 



(17) 



8 2 = — I fc' (7 - AA + A' + A£-A + £,A') & 
1 f°° 

= <^ a „ + — b'AE'^EiA'b dco 
1 f °° 

= 5 /«;;+^- fc'AA+C-Dr'qA+A'fedw 

J -co 
1 f°° 

where C •!/«;; is the vector of the elements of T/ M // that 
are constrained in the i™ Boolean structure. Equation ( TTTb 
expresses the optimum 6 value of every Boolean structure in 
terms of the 6 value of the fully-connected structure plus the 
effect of constraining some of the links to be zero. 

The matrix D, is block-diagonal and composed of elements 
of A + , which is a block-diagonal matrix given by A + = I ® 
(G'G) 1 , where I is the p x p identity matrix. If no more 
than one element is constrained in each column of X, then 
Dj is diagonal and composed of diagonal elements of A + . In 
this special case, (fTTI i reduces to: 

for all elements j of Y that are constrained in the i ,h structure. 



B. Proof of Lemma \3\ 

Proof. Since X, has only one zero element, Lemma|2]results 



i r c 

5^-6%^-- I 



\Y full (j)\ 2 
A + (j,j) 



dco 



(19) 



where j is the index of the constrained element in Yf u u. 
Since G = (I + eAo)G,, A can be written as A = (7 + 
(7 <g> eAo))A,, where A t - I ® G t . Due to the equivalence 
of dynamic uncertainty models it is possible to parameterise 
the uncertainty in a number of forms, which allows us to 
write: (A'A) -1 = (A' t A t y l (I + Aj(e)) for some Ai(e) which 
is also a function of Ao and A,. Similarly we can write: 
(A'A)-'A' = (A;a,)-'A;(7 + A 2 (e)) for some other A 2 (e). 

Denoting A, + = (A' t A,y l , A, = A+A' t and A = A + A', the 
integrand of (fT9] l is given by: 

\Y fu „(j)\ 2 \A(j,:)b\ 2 



A + (j,j) A+(j,j) 



\A,(j,:)(I + A,)b\- 
A ( + 0;:)(7(:,;) + A 2 (:,;)) 



(20) 



\A,(f:)b\' 
At(JJ) 



(l-aj(e))+Bj(e) 



where (j,:) and index respectively the /* row and 

column of a matrix, and the dependence of Ai and A 2 
on e has been omitted. The functions aj(e) and Bj{e) are 
continuous functions of e, satisfying c/0) = and /8/(0) = 0. 

Note that 6f(0) = ± rfw and hence £[9) can be 

written: 



<5?(e)-4„„(e) = ^(0) + /i(e) 



(21) 



where /1(e) = ^ jZ,^j( € ) ~ ^'ojj ^ w is also a 

continuous function of e and satisfies /-(0) = 0. ■ 



C. Proof of Lemma [4] 

Proof. Define Yj as Y with every element that is constrained 
to be zero in the f h Boolean structure removed. Similarly 
Aj is obtained from A by removing the columns of A corre- 
sponding to the constrained elements of Yj. Taking Yj as the 
unconstrained minimising argument of 6 2 = inf f \\AjYj—b\^, 



Lemma [3] gives: 



\Yj(k)\ 2 

-co A j (/c, /c) 



dco 



(22) 



where k is the index of the constrained element in Yj - 
A + A'.b. Recall that A, 



I <g> G, and obtain A, from A, by 
removing the columns of A, corresponding to the constrained 
elements of Yj. Then we can write Aj = (7 + (7<g>eAo))A, and 
this problem is now in the same form as that of Lemma [3] 
and the proof follows as before. ■ 



D. Proof of Theorem Q] 

Proof. A problem is M2 solvable from Lemma Q] if for every 
allowable structure Y a and every non-allowable structure Y n 
with the same number of links, 6 2 {e) > 61(e). From Lemma 
|U for every allowable structure Y a , 5 2 a {e) is given by: 

8l(e)=6 2 ai (e)+f a , ai (e) 

= 5 )uu^) + /«.«, ( f ) + X •^'• fl ' +1 (e) (23) 

= 4^) + ^) 

where a, denotes an allowable structure with i links more 
than Y a , from which y a can be obtained by removing links. 
Here we have used the facts that any allowable structure can 
be obtained by removing one link from another allowable 
structure and that <5 fl (0) = for all allowable structures due 
to solution uniqueness from CorollaryQ] From Lemma|H the 
functions fa h a M {c) are continuous and satisfy / a; , a;+1 (0) = 0, 
hence F a (e) (which is a sum of these functions) also satisfies 
these properties. 

Similarly, for every non-allowable structure Y n : 

£2(6) = £2(0) + + /„,„,(£) 

= 2 6 2 ni (Q) + 6%„(e) + J] fn t ,n M (e) (24) 

i i 

= B„ + S 2 full (e) + F n (e) 

where n, denotes a non-allowable structure with i links more 
than Y„, from which Y„ can be obtained by removing links. 
The constant B„ = 2/^,(0) > again due to solution 
uniqueness from Corollary Q] and, as in the allowable case, 
F n (e) is continuous and satisfies F n (Q) = 0. The solvability 
condition is then: 

5 2 n (e) - S 2 a (e) = B n + F n (e) - F a (e) > (25) 

where the difference between 5 n and S a is characterised by a 
perturbation from each of their nominal (e = 0) values. From 
the properties of F„ and F a , the function F„ a = F„-F a is also 
continuous and satisfies F na (Q) = and hence there always 
exists some r m > such that \F na (e)\ < B n for all e < r„ a . 
The solvability condition of d25l ) is then satisfied as follows: 

Sl(e) - 5 2 a (e) = B„ + F„ a (e) 

> B„ - \F„(e)\ (26) 

> 

for all e < r na . Taking r to be the minimum r„ a over all 
pairs of allowable and non-allowable structures will therefore 
ensure the problem is M2 solvable for all e < r. m 



